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Citation counting has been widely discussed in Britain in 
recent years, but rarely in any detail. It has emerged onto the 
scholarly agenda because it is seen as a plausible and simple 
means of measuring academic performance. Jack Meadows has 
defined it recently as follows: 

Citation counting is part of bibliometrics and involves 
the quantitative study of the references appended to 
documents (usually articles in journals) (Meadows 1991). 
This paper reports on an empirical study of two citations 
analyses, carried out on two British scholarly periodicals in 
the field of continuing education for the years 1986 - 1990. 
Our findings confirm the widespread scepticism surrounding 
proposals to use citation counts to measure quality; we think 
them more worthless for that purpose than we did when we began 
our inquiry. At the same time, citation counting did help us 
produce a "sketchmap" of the current state of scholarship and 
research in British continuing education, and it is in this 
latter respect that it may have a valuable if limited role. On 
the assumption that some for of citation counting is probably 
inevitable, given the present state of British higher education 
and its managers, we conclude with some brief recommendations 
for the two main scholarly bodies in British continuing 
education. 



Introduction 

Continuing education is, in Britain, far better established as 
a field of practice than as an area of study. As a field of 
study, continuing education developed in the twentieth century 
from fairly modest origins, and it remains very much an applied 
field of study, whose concepts are drawn from other parent 
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disciplines (sociology, economics, history, psychology, 
philosophy and politics); where knowledge is derived from the 
field; and where research and scholarship are larg :^ly treated 
as subordinate to practice (Jarvis 1990) • Further, the numbers 
of British scholars involved in continuous and sustained 
research are relatively small. 

As an area for research, then, continuing education is probably 
best regarded as a subset of wider fields, such as educational 
studies or management. Its institutional conditions of 
existence, tnough, are rather different. Unlike mainstream 
management or educational studies, which are taught and studied 
in universities, polytechnics and colleges but are at least a 
coherent domain of research, the study of continuing education 
is dispersed. Teaching and research into adult education is 
concentrated mainly in the universities; that into further 
education tends to be located in the polytechnics; nursing and 
management education can be found in both; there is little or 
no teaching and research into training and human resource 
development. Continuing education's split institutional 
identity sets it apart, then, from mainstream educational 
studies. 

Within the universities, still at present the main source of 
published scholarship, continuing education has a long extra- 
mural history. It existed as a field of practice for some fifty 
years or more before it also became a field of teaching and 
research, and there is a strong predisposition to regard the 
former as the primary function. Historically, organisational 
and cultural factors both favoured a low status for research 
into CE; most people did not even regard themselves as adult 
education specialists at all. Where specialised CE Departments 
existed inside universities , lecturers have constructed for 
themselves an identity which is rooted partly in their practice 
(on the margins of the University) and partly in the discipline 
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which they teach to their adult classes (a varied clutch, 
including management, botany, archaeology and women's studies). 

As a result, research into the education of adults has only 
been carried out by a minority within the Adult Education 
Departments. That minority has often felt itself embattled; its 
public vehicle, the Standing Conference on University Teaching 
and Research into the Education of Adults (SCUTREA), ploughed a 
lonely furrow for many years. Some CE researchers turned to 
research methodologies - participatory research, for example - 
which appeared congruent with the values and practice of CE 
more widely, but were not always recognised as research by 
scholars from more conventional disciplines. 

In recent years much has changed. Old structures of University 
adult education have collapsed; as they have started to operate 
more within the University mainstream, then if only for reasons 
of survival and credibility, the remaining Adult Education 
Departments have to be seen to do research, and do it well. 
Short-term earmarked funding is available for CE research from 
the Universities Funding Council, on a competitive basis. Our 
entry into European networks, often through Community 
programmes surh as ERASMUS, has alerted many of us to the 
relatively poor standing of British University research into 
the education of adults in the eyes of many of our French, 
German or Dutch colleagues. Research has, as a result, a new 
salience within most Adult Education Departments. 

This has not been a conflict-free process. Competitive bidding 
to the UFC for funds has brought rivalries, and the selective 
outcomes have meant disappointment for many. The very idea of a 
mainstream Adult Education Department is offensive to many who 
were brought up in the Extra-Mural tradition, as is the idea of 
appraising the research performance of a lecturer who was hired 
in the 1960s to teach local history or the Victorian novel. It 
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is easy to see why the idea of measuring research quality - 
hotly contested as it is throughout the University system - has 
caused so much anxiety in continuing education, manifested in 
reactions to the University Grants Committee's last selectivity 
exercise as well as in the muddle in which the UGC's CE panel 
got itself. 

In these circumstances, should we try and measure the quality 
of research in CE, and if so how? Measurement is, after all, 
usually deemed a necessary precondition for accounting, judging 
and improving performance. It is also of increasing importance 
in the distribution of resources: research selectivity review 
ratings are already used to allocate recurrent grant, and UFC 
has indicated that they V7ill be of growing significance in the 
future (with universities being asked whether they wish to be 
told how the research part of their grant divides up as between 
cost centres). Measuring quality is also a useful exercise in 
its own right, enabling us to compare the relative performance 
of institutions, and to identify i^hat kinds of institutional 
support and academic planning best promote good research 
(Baumert & Roeder 1990). 

No single means of measuring research quality has yet won 
widespread acceptance. In the past, the UGC judged research 
performance by a process of peer review. Inevitably, this 
process attracted criticism: a member of the AUT executive, for 
exa'^ple, suggested that the UGC panels were "affected by 
personal prejudices", and were London-centred so that 

The outcome suggests a 'miles from Park Crescent' inverse 
factor somewhere in the rating formula (Emanual 1989: 8). 
Subjective judgements are always likely to attract criticism, 
precisely because they are subjective; the question remains 
whether more objective and quantitative measures are any more 
accurate than the old-boyism that characterised che UGC. 
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Citations analyses represent the most commonly used attempt to 
give a quantitative value to research quality. This paper 
reports on an analysis of citations in articles in the two most 
prestigious British journals in the field, Studies in the 
Education of Adults and the International Journal of Lifelong 
Education, between 1986 and 1990, Our intention was twofold. We 
wished (1) to evaluate the potential of citations analysis - 
the main quantitative method for studying the quality of 
research output - and (2) to see whether citation counts would 
help establish the extent to which British adult education 
researchers are creating a distinctive body of knowledge. 



Citation Counts 

The citation count is not yet widely used in British academic 
circles, and may therefore be unfamiliar to many researchers. 
In the United States, however, citation counting in its 
contemporary form has gone on at least since the publication of 
the Science Citation Index in 1961. Originally developed for 
the purposes of information retrieval and document tracing, 
citation counting has also been used bibliometrically to 
develop rankings of journals, publishers and authors in various 
fields; this information in turn can be used to identify 
patterns of intellectual influence within disciplines, map the 
existence of "invisible colleges", and trace the diffusion of 
new concepts, discoveries or empirical findings (Smith 1981). 

Citation counting is also sometimes used in the US as a 
quantitative indicator of quality in judging published research 
output. For example, some American universities use it to 
determine lecturer and researcher productivity when reaching 
decisions on tenure, promotions and salary merit raises. 
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Althoush this is probably not its main use, it is certainly 
what most British academics think of when they hear of it. The 
assertion made by defenders of this practice is that the 
citation count offers an accurate proxy measure for quality 
because it represents the impact of a publication on the work 
of informed peers. This rests on a number of assumptions, all 
of which have been challenged on bibliometric grounds. 

Most straightforwardly, it has been alleged that citation 
counting is empirically flawed. Weaknesses include the poor 
reliability of some authors' citations in the literature being 
surveyed: one study of two prestigious psychology periodicals 
showed that 11% and 14% respectively of all citations were in 
error; other studies have shown that, once a source is cited 
wrongly, the error will be repeated by other authors (Sweetland 
1989: 295 - 300). This is essentially a practical problem 
(Sweetland recommends improved training in research methods); 
more complex is the assumption made of authorial intentions. As 
Michael and Barbara MacRoberts demonstrate, authors frequently 
do not cite their main influences; nor are the most frequent 
citations necessarily positive ones (MacRoberts and MacRoberts 
1989). Epistemological differentiation between different 
domains of knowledge, or straightforward socio-cultural bias, 
may also play a part - in the form, for example, of an apparent 
gender effect in citation patterns (with male scientists being 
more likely to be cited than women, providing "proof" to the 
naive that women produce less significant research (Cole 1979: 
64)). 

Such weaknesses are well-known in the US. In Britain, the 
Universities Funding Council concluded on the basis of a pilot 
study in science and engineering that 

there was little to be gained by the use of citation 
counts and, more broadly, that bibliometric analysis was 
as yet a very poor substitute for peer review (UFC 1989: 
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14). 

However, UFC was also aware that the process of "peer review" 
is extremely subjective and impressionistic, and is open to 
criticism on the grounds that the criteria used in determining 
relative degrees of "excellence" were not clear to 
participants. Similarly, a report on peer review to the 
Advisory Board for the Research Councils concluded that 
although 

using such indicators in micro-level assessment is not 
generally accepted as either reliable or useful • . . 
where such information is easily available , it should be 
provided for peer review bodies, which can then assess its 
value for their specific purposes (Working Group on Peer 
Review 1991: 5-6). 
In short, the jury is still unconvinced that citation counts 
can help measure individual performance; but is prepared to 
countenance their use in judging the va?.ue of research 
proposals. Assessment of performance, preferably through 
quantitative indices, is then increasingly accepted in policy 
circles as an essential component of "accountability". 
Citations analyses have their supporters, and from time to time 
they will be proposed as providing an alternative, and 
reliable, measure of quality. 



Continuing Education and Citations Analysis 

Continuing education as a field of study does not lend itself 
to quantitative measurement, particularly on the basis of 
citations counting. First, it is poorly represented within the 
Index. Second, all educational research tends to be driven by 
normative and humanistic assumptions; qualitative methodologies 
are generally preferred over quantitative, which are seen as 
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"reductive" and "positivis tic". In continuing education, the 
scene is complicated by the characteristic structure of the 
University Department of Continuing Education, which might be 
expected to be - and in fact is - the main basis for CE 
research. 

Structurally, the decisive fact is that the nature of the field 
is driven by practice. University CE Departments until the 
1980s generally consisted of scholar-practitioners who were 
employed primarily as animators or as teachers in other 
disciplines (history, drama, life sciences, etc). With a very 
small number of exceptions (Leeds, Nottingham, Manchester) 
their research act'^^^ities rarely focussed upon building a 
theoretical knowledge base in the field of adult education. In 
fact, some lecturers experienced the absence of a 
pretentious/precious "research culture" as one of the most 
positive and joyously emancipatory aspects of the job. 

This is partly a practical problem, which of course to an 
extent applies to similar fields such as social work, business 
studies, some forms of engineering, and educational studies 
more widely. But the argument also has an epis temological 
dimension, perhaps uniquely so in CE. Teachers in any applied 
subject could argue that if knowledge is designed to be useful 
to practitioners and policy-makers, then surely it makes more 
sense to measure its impact amongst these rather than upon 
fellow-researchers. In CE, though, and particularly in adult 
education, the 1970s witnessed the flowering of alternative, 
critical and populist concepts of research that were congruent 
with the libertarian educational practices associated with the 
writings of Paolo Freire. The most significant of these - 
participatory research - was explicitly anti-scholarly, anti- 
academic and a- theoretical , seeing the "expertise" of the 
University as part of the problem which popular educational 
movements sought to overcome (Hall 1975: 25-8). From this 
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point of view, measuring the quality of academic research is a 
completely pointless activity. 

For CE Departments, then, the very nature of scholarly research 
is itself a contested issue. Developments since the early 1980s 
have done much to change the context in which research is 
undertaken. Internal developments favour the growth of a 
"research culture" within CE Departments: positively, the 
practice of CE has moved to a more central and less marginal 
position within the University; negatively, what Harold 
Wiltshire called "the Great Tradition" of liberal adult 
education has largely lost ground. Earmarked funding from the 
UFC, the recognition that European colleagues have sometimes 
developed significant scholarly contributions, and increased 
government pressure for selective funding according to 
performance have all raised the salience of research 
performance within University CE Departments. Yet for most CE 
researchers, quantitative measurement of research quality 
through citation counting is still seen as a foreign import, at 
best problematic, at worst actively damaging and reductive, 
which threatens the distinctive status of adult education as a 
combination of field of practice and field of study. 

Epistemological reflection is, though, largely beside the 
point. From the existing sources, it is impossible to carry out 
a credible analysis of citations in continuing education. The 
primary source for citations counting in educational studies is 
the Social Science Citations Index, published annually by the 
US-based Institute for Scientific Information. SSCI is compiled 
by scanning yearly the citations of over 1,400 journals; of 
these, there is a total of 87 journals in the category of 
Education and Educational Research (and 20 more in the Special 
Education category). Of these, six deal with higher education, 
one with educational gerontology, and only one with the 
education of adults (the American Adult Education Quarterly). 
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in other words, continuing education as a field of study is 
seriously under-represented in the Index (and vocational 
education in particular is missing entirely). No reliable 
judgement could conceivably be formed of the merits of British 
scholarship using the SSCI as it now stands • 

In itself, again, this is a practical problem. It could easily 
be overcome by adding a few appropriate journals to the Index. 
We have therefore attempted to replicate, in the British 
context, research carried out during the 1970s on the American 
journal Adult Education (Boshier and Pickard: 1979). We 
examined every citation made by authors of all substantive 
articles (not book reviews) published in what we believe are 
two leading British scholarly periodicals (the International 
Journal of Lifelong Education (IJLE) and Studies in the 
Education of Adults (Studies)) in the five year period between 
1986 and 1990. Both are refereed journals, edited by respected 
scholars; although IJLE has the more prestigious editorial 
board by conventional standards, board members are in both 
cases identified by their University; Studies has the 
sponsorship of SCUTREA as well as of the Natiional Institute of 
Adult Continuing Education. In other words, these are precisely 
the kinds of journals that might be scanned for the Social 
Science Citations Index. 

Our intention was to: 

1. identify those writers and works cited most frequently; 

2. study the relative influence of "primary", "secondary" and 
"tertiary" literature on authors; and 

3. compare findings across the two journals. 

Subsequently, we hoped to identify some of the broad trends and 
prospects in the field. 
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Approach 

Every reference cited by authors of all articles was listed on 
index cards. This gave a total of 2,353 references, each of 
which was classified according to its main subject focus. It 
was then allocated to one of three main categories: primary, 
secondary or tertiary literature. The guidelines used in 
categorising references were taken and amended from Boshier and 
Pickard: 

(a) Primary literature = concepts, processes and data clearly 
identified with adult education. It is usually published 
in adult education books, journals or monographs; it is 
produced by people for whom adult education is their 
primary professional concern. It could include further 
education, vocational training and higher education - even 
schools in some circumstances - but only in so far as the 
reference deals with their work with adult learners. 

(b) Secondary literature « concepts, processes and data 
identified with educational studies, but not with the 
education of adults strictly defined. It is usually 
published in education books, journals or monographs; it 
is produced by people for whom other sectors of education 
(schools, vocational training, higher education) are their 
primary professional concern. 

(c) Tertiary literature « concepts, processes and data not 
primarily identified with education at all but with some 
other field of study. It is not usually published in 
educational journals or books, but is cited by the author 
because it deals with or supplies concepts, processes or 
contextual data which act as points of reference. 

This classification was intended to provide a somewhat more 
refined analysis than that of Boshier and Pickard, whose 
analysis of citations in the American journal Adult Education 
used only two categories (primary adult education literature 
and other literature) (Boshier and Pickard 1979). We wanted to 
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know whether the wider field of educational studies has made 
any impact upon scholarship in continuing education, and so we 
included the intervening category. In the vast majority of 
cases, there was no difficulty in assigning a reference to the 
appropriate category. In some cases, where there was a degree 
of ambiguity, the reference was assigned to the category that 
appeared most appropriate after discussion. 

As a result of the exercise, we analysed a total of 2,353 
references cited. Far more of these came from IJLE than from 
Studies: 1,752 for the former, as opposed to 601 for the 
latter. We did not weight for this discrepancy. Rather, we have 
presented the data for the two journals separately in our 
tables . 



Findings 

Our work was designed to (a) test the likely impact of citation 
counting in continuing education; and (b) see what the exercise 
told us about the nature of knowledge in the field. We provided 
four main quantitative assessments of the total of citations. 
We have identified: 

(a) the most frequently cited texts, using a simple aggregate 
count of references (Table One); 

(b) the twenty most frequently cited authors, using a "full 
publication equivalent" which allocates one point for a 
single-authored text and half a point per author for 
reference to a jointly-authored text (Table Two); 

(c) the pattern of self -citation among the top twenty; and 

(d) a break down of all references showing the balance between 
primary, secondary and tertiary literature (Table Three). 

We have not weighted as between books , edited collections or 
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articles (a weighting scheme is proposed in Furnham 1990), Nor, 
in aggregating the total counts, have we weighted for the 
differential citation patterns of the two journals; however, we 
rapidly found that the total number of references in Studies 
was far smaller than that in the International Journal, and 
readers may wish to bear this in mind, 

Paolo Freire and Malcolm Knowles were the most frequently-cited 
authors, followed by Roger Boshier, Jack Mezirow, Stephen 
Brookfield and John Mcllroy, Of these six authors, three are 
north Americans, one a Latin American, one a New Zealander, and 
one - Mcllroy - British. North American domination is less 
marked in the bottom than in the top half of Table Two: seven 
out of the top ten are North American scholars, but only two of 
the following ten. Of the three mainland Europeans among the 
twentyfive most frequently cited authors, only one (Ettore 
Gelpi) is still alive; the other two - Karl Marx and Antonio 
Gramsci - are what you might call scriptural authors. 

We found a marked gender imbalance. Among the twenty most cited 
authors, eighteen are men and two are women (Patricia Cross, an 
American, and Jane Thompson, the only British woman writer to 
feature in the top twenty). This casts doubt on the view that 
citations counting is less biassed against women than peer 
review; it is probable that the old-boy networks which operate 
strongly throughout British higher education are as likely to 
be reflected in citation patterns as in other qualitative 
judgements. 

Self-citation is marked among at least five of the twenty most 
frequently cited writers. Self-citation accounted for all the 
references to Bagnall and Nordhaug, well above half of the 
count for Mcllroy and Usher, and just over half for Boshier. Of 
these four, only Boshier would still have featured in the top 
twenty had self-citation been excluded from the count. Neither 
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of the two women authors were present as a result of self- 
citation. 

Most of the frequently-cited texts are primary adult education 
literature. They also tend to be books. Freire's The Pedagogy 
of the Oppressed, first published in English in 1970, was the 
most frequently cited reference in both journals. In Studies it 
was followed by a work on the history of ideas (Kuhn's 
Structure of Scientific Revolutions), a book on action research 
(Carr and Kemmis ' Becoming Critical), and R. W. K. Paterson's 
Values t Education and the Adult. In the International Journal, 
Freire was followed by the Faure report (Learning to Be), a 
reflective report on field work (Tom Lovett's Adult Education, 
Coamunity Development and the Working Class), Knowles' The 
Modern Practice of Adult Education, and Patricia Cross' Adults 
as Learners. All of these oft-cited works are books rather than 
articles; only one (Kuhn, though Kerr and Kemmis might be 
considered as spanning the boundaries of educational studies) 
is not primarily concerned with the education of adults. 

Research rarely appears among the most frequently cited works. 
Carr and Kemmis' Becoming Critical is a methodological 
textbook; it is about how to do action research, rather than 
presenting the results of some inquiry. Accordingly, our 
findings suggest that citation counting is not likely to prove 
a reliable means of measuring the quality of published research 
in the field of continuing education. This was as we had 
anticipated. What we had not realised, though, was the sheer 
extent to which citation is an unreliable proxy for quality. 

Much of the work cited had no connection with current research 
into continuing education. The most frequently cited text - 
Freire' s Pedagogy of the Oppressed - is inspirational so far as 
practice is concerned, but is only marginally concerned with 
research. Several of the authors are long deceased: Karl Marx, 
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for example, was one of the most frequently cited writers in 
our survey. Other frequently-cited works - e.g. Kuhn's The 
Structure of Scientific Revolutions - come from quite separate 
fields of study from continuing education. Table Three makes 
clear the general dominance in both the journals of concepts, 
data and processes drawn from what we defined as Secondary and 
Tertiary Literature. It also suggests that the number of 
citations per issue is rising steadily; and that the role of 
tertiary literature is growing. This trend is particularly 
marked for Studies, which acquired a new editor during our 
period, an actively publishing CE historian whose stated 
intention was to raise the journal's academic standing. 

The main type of theoretical tertiary literature cited was 
political (with Marx leading the field, and Gramsci some way 
behind). Works of general social or economic theory were rarely 
cited; the works of Juergan Habermas, for example, were cited a 
total of five times (twice in Studies, three times in IJLE) ; 
interestingly, most authors citing Habermas come, like the 
Canadian Michael Welton, from overseas; this was not enough to 
bring him into any of our Tables. This pattern seems to reflect 
a generational culture: if they acknowledged any macro- 
theoretical influences at all, most British CE scholars during 
the late 1980s were more likely to cite the works of Marx and 
the more humanistic of his followers (Gramsci in particular) 
which had influenced them in the 1960s and 1970s than any 
contributor to contemporary social theory. But this should not 
be overstated. References to tertiary literature, although 
numerous in aggregate, tended to be dispersed among many 
authors, reflecting the multidisciplinary nature of CE as a 
field of practice as well as of study. 

Primary references tended to concentrate more heavily around 
single authors. This is not to say though that a coherent 
theoretical debate is emerging among British CE scholars. 
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Rather, the most frequently cited texts tend to be 
prescriptive. Apart from Freire (10 citations in the 
International Journal, four in Studies), frequently-cited texts 
included the 1972 UNESCO report. Learning to Be; two books by 
Malcolm Knowles; and Patricia Cross' Adults as Learners: 
increasing participation and facilitating learning. Work 
directly involving research as such was rarely cited; Kerr and 
Kemmis is exceptional, but was as we have said cited largely 
because of its methodological significance; it acts as a 
"cultural signpost". In the International Journal, Tough's 
study of independent learning projects, Kelly's historical 
account of British adult education and the now rather dated 
participation study by Johnstone and Rivera received the 
highest number of citations for research-based studies (five 
each) . 

What we did not find are high levels of citations for the 
contemporary research of individuals. If self -citation is 
excluded, the only British author in the top ten is Ken Lawson, 
a philosopher; the next is Peter Jarvis. Neither is a current 
researcher, nor are they cited for empirical research. Nor is 
it possible to use our citation count to judge the quality of 
work emerging from the leading British CE Departments: Leeds, 
which we would ourselves judge the most productive and 
influential of British CE Departments during the 1980s, simply 
does not register as a source of quality research. The fact is 
that empirical research is not cited nearly as often as texts 
which are prescriptive, which summarise the state of 
scholarship in a convenient and compact manner, or which 
provide a source for a seminal but much-disputed "concept" like 
andragogy. 

Attempts to measure quality through citations counting will, 
then, be entirely unsuccessful for the field of continuing 
education. Scholars will persistently challenge the findings of 
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any exercise which is based on the assumption that frequent 
citation is a measure of excellence; far from having any 
legitimacy, the exerc3.se would be regarded as largely 
meaningless. Yet this is not entirely the case; although 
unreliable as an indicator of quality, citations analysis does 
reveal important patterns of influence and provides elements of 
a map of the field of study as it currently exists. 



Defining the Research Territory 

The findings of this exercise suggest a number of disturbing 
conclusions about the nature of British CE research. In 
particular, they indicate that the study of continuing 
education is still in a formative and under-developed stage; 
there is little if any sustained dialogue and debate among 
continuing education scholars; there are no widely shared 
conceptual frameworks of analysis, and those that exist are 
largely derived from systems of thought whose engagement with 
the field of adult education is rarely systematic. North 
American literature has a quite disproportionate influence; 
although there is some interest in and connection with European 
research, it has yet to bear fruit* 

Sustained dialogue amongst scholars is commonly seen as the 
means whereby a discipline or field of study is formed (Elliott 
1990). It fosters the development of a common discourse of 
analysis ; it provides a terrain where alternative conceptions 
and explanations are advanced, clarified, tested and 
interrogated; and it allows interested parties to define the 
problem-fields for further study. A grasp of these debates is 
commonly used to induct younger scholars , and presumably it 
also serves a continuing career development function for 
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individuals. It is clear from our analysis of citation patterns 
that no such sustained dialogue is currently detectable among 
British students of continuing education - at least, not on any 
scale. 

Evidence of a sustained engagement with one or more clearly 
defined problem-fields is absent from our study. Quite the 
reverse: for example, there is little cross-citation between 
John Mcllroy and Roger Fieldhouse, currently the two leading 
historians of workers' education in Britain. There are signs 
that some common problem-fields are emerging: participation 
research is one such area; high citation counts for Brookfield 
and Tough may suggest that self-directed learning is an 
emergent shared preoccupation for some scholars; there is also 
a debate of sorts over the concept of andragogy. There is 
nothing to parallel the "standard of living debate" among 
historians, "the question of class" among sociologists, or "the 
unhappy marriage of marxism and feminism" debate between 
feminists some years ago. There is very little sign of a 
textual conversation; what we see is a group of people 
apparently bent on speaking past one another. 

This problem is compounded by an evident discontinuity of 
authors. As in the USA, much writing in continuing education 
seems to be "single-shot" publishing: that is, the author 
writes one or perhaps two articles, then vanishes. This is 
common enough in practice-driven fields, where much of the 
writing is by practitioners who undertake a brief period of 
study - often towards a higher degree - then return to the 
field. Sadly, this practice is probably about to enter a period 
of decline; partly for generational reasons (there is little 
turnover among CE practitioners) and partly for fiscal ones 
(particularly the demise of "pooling" arrangements, whereby 
year-long secondments were funded by central rather than local 
means), sustained pieces of practitioner inquiry are likely to 
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become rarer. As a result of this development, and the new 
salience of research output within the University CE community, 
we suggest that publications such as Studies and IJLE will come 
increasingly to resemble conventional academic journals, whose 
function is largely that of providing space for debate among 
University CE scholars (and, in case it needs stating, thereby 
serving individual and institutional advantages). 

What kinds of research might arise from the growing 
"conversational community" in CE? It is likely that the 
largely normative approach will continue, with most scholars 
assuming that the education of adults is something which should 
always be encouraged, and that research should help meet this 
goal. It is likely steadily to lose the humanistic values which 
fostered support for the idea of participatory research in the 
1970s; generational changes, and the growing importance of 
continued vocational education within Universities, present a 
substantial challenge to the humanistic ethos of University CE. 
Partly but not entirely for linguistic reasons. North American 
models may continue to exert an undue methodological and 
topical influence: for example, participation research has made 
limited impact in the UK, in contrast to the USA, but it is 
probable that it will move to a more central place. North 
American scholars have for two decades studied the patterns of 
participation by adult learners and tried to establish their 
motivation for doing so, partly under pressure from the field. 
As British continuing education becomes more market-oriented, 
it is possible to predict that the American research will 
attract even greater interest. Johnstone and Rivera's 
Volunteers for Learning (first published in 1965) was cited 
five times in the International Journal, where Roger Boshier 
was also the most frequently-cited individual (his 1971 article 
on motivation was prominently referenced). 
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As European networking spreads among CE researchers, so a 
further and probably separate trend may emerge which is 
oriented more towards theory and concept-building than 
practice. However, our citation search has not thrown up any 
evidence that this is happening yet. American intellectual 
hegemony is , on our evidence, secure. Citation counting is , 
though, an intrinsically historical exercise. It analyses the 
footnotes to articles published some years ago, and written 
maybe two or three years before that. In other words, our 
evidence tells us more about the early than the late 1980s. 



Conclusion 



We are concerned in this paper to identify the implications and 
explore the possible uses of citation counts in continuing 
education in Britian. We have emphasised the limitations to 
citation counting as a means of measuring the quality of 
research and scholarship in continuing education. Yet we are 
still able to draw a number of conclusions concerning the 
nature of CE as a field of scholarship in Britain: namely that 
it is often conducted by isolated individuals rather than 
conversational communities, it is unduly influenced by a small 
number of largely North American authors, and that its most 
significant scriptural figures tend to reflect the normative 
and humanistic ideals of contemporary British scholars in the 
field. Although there is a growing reliance on what we call the 
"primary literature" of CE, this does not provide evidence of 
"invisible colleges" of scholars pursuing common debates and 
themes. CE research is then weak in a number of key respects, 
and is therefore vulnerable to invasion from scholars working 
in other fields of study (industrial relations, organisation 
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studies, sociology of race and ethnicity, even educational 
studies) • 

Our use of citations analysis has been, then, to examine the 
state of the art. To say that we doubt its value as a means of 
judging research quality is something of an understatement! Yet 
we do take it seriously, believing that the importance of 
citation counts is almost certain to grow, for several reasons. 
Current developments are rapidly enhancing the ease with which 
bibliometric studies are carried out. Information technology 
and information science have greatly increased the* amount of 
bibliographic detail that can be electronically stored; 
information from the Institute of Scientific Information 
Indexes is now available on-line in all British Universities, 
through the BIDS consortium. This development affects one of 
the main objections to citation counting, namely that, as the 
Advisory Board's report noted, 

collecting information on outputs can be difficult and 
costly, and the current techniques which attempt to 
analyse quality are expensive (Working Group on Peer 
Review 1991: 12), 
As the process becomes easier and cheaper, so the practical 
obstacles to citation counts will diminish. 

Citation counts are, we have found, not an effective and 
accurate way of measuring the quality of research, nor even of 
scholarship, in continuing education. It would though be 
sensible for SCUTREA and the Universities Council on Adult and 
Continuing Education (UCACE) to recognise that they may well be 
introduced, at least in some British Universities, at some time 
in the future. Pressures for greater accountability, and T 
more f ocussed linking between performance and resourcing 
(corporate and individual) , are so intense that even daft 
solutions sometimes seem attractive. This being so, it makes 
sense for our representative bodies, UCACE and SCUTREA, to take 
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citation counting more seriously, and maybe even to consider 
how the existing bibliometric system needs to be changed in 
order to limit the damage. 

We therefore suggest that UCACE and SGUTREA: 

1. use their relationship with their sister organisations in 
Canada and the United States to identify the ways in which 
citation counts are currently used and what the effects 
have been; 

2. consider their own policy on the use of citation counts as 
proxy measures of quality in scholarship and research in 
continuing education; 

3. raise the question of citation counts with other scholarly 
bodies in the social sciences (and especially the British 
Educational Research Association); and 

4. approach the ISI with a request that a small number of 
appropriate journals be included in the Social Science 
Citations Index (not least so that BIDS becomes a more 
worthwhile resource for CE scholars). 

We see this as little more than a basic damage limitation 
exercise, though a necessary one. What is more important is 
that it be linked to longer term measures to raise the quality 
of continuing education research as a whole in Britain. 
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TABLE ONE; MOST FREQUENTLY CITED PUBLICATIONS 



Numbers of citations (FPEs) in the two journals 

Total 



Freire, Pedagogy 
of the Oppressed 



Studies 
4 



Paterson, Values, 3 
Education and the Adult 

Cross, Adults as 2 
Learners 

Faure, Learning 0 
to Be 

Knowles, The Modern 1 
Practice of Adult 
Education 

Kuhn, Structure of 4 
Scientific Revolutions 

Lovett, Adult 1 
Education, Community 
Development and the 
Working Class 

Griffin, Curriculum 2 
Theory. . . 

Knowles, The Adult 2 
Learner 

Carr & Kemmis, 3 
Becoming Critical... 
Through Actioii Research 

Houle, Enquiring 2 
Mind 

Johnstone & Rivera, 1 
Volunteers for Learning 



Kelly, History of 
Adult Education 

Mezirow, Critical 
Theory of Adult 
Learning 



1 

0 



IJLL 
10 

7 

6 

8 

6 

3 
6 

5 
4 
3 

4 
5 
5 
5 



14 
10 
8 
8 
7 

7 
7 

7 
6 
6 

6 
6 
6 
5 
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Mezirow, Critical 
Theory of Self- 
Directed Learning 

Berger & Luckman 
Social Construction o^ 
Reality 

Brookf ield , Under- 
standing and Facil- 
itating Adult Learning 

Jarvis, Adult and 
Continuing Education 
Theory and Practice 

Lawson, Philosophical 
Concepts and Theories 
in Adult Education 

Tough, The Adult's 
Learning Projects 



1 4 5 

2 3 5 
1 4 5 
1 4 5 
1 4 5 
1 4 5 



Note: to be included in this Tf.ble, a publication had to score 
at least five FPEs in total, with a minimum threshold of either 
two FPEs in Studies or four FPEs in IJLE, the differential 
reflecting the more frequent publication scheduling of the 
latter. 
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TABLE TWO: MOST FREQUENTLY CITED AUTHORS 

Numbers of citations (FPEs) in the two journals; 

Studies IJLE Total 
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14 
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0 


1 
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Peters, RS 


1 


12 


13 


Marx, K 


7.5 


5 


12.5 


Jarvis, P 


4 


8 


12 


Houle, CO 


4 


6 


10 


Erikson, EH 


2 


8 


10 


Thompson, J 


3 


6 


9 


Griffin, C 


4 


5 


9 


Raybould, S 


3 


6 


9 


Paterson, RWK 


3 


6 


9 


Gelpi, E 


1 


8 


9 



* We excluded 19 references to Raymond Williams' work, since 14 
were in an article marking bis death. 



TABLE THREE: REFERENCES TO PRIMARY, SECONDARY AND TERTIARY 
LITERATURE, 1986 & 1990 

Primary Secondary Tertiary TOTAL 

(a) Studies, vols 18 and 22 

1986 46 24 8 78 

1990 62 41 63 166 

(b) IJLE, vols 5 and 9 

iS^oo 131 96 134 361 

1990 153 167 161 481 
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